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ABSTRACT 

This study sought to determine the levels of 
persistence in scanning postings that are typical of users of one 
online public information system, LCS (Library Computer System) , as 
it was configured in Illinois in 1987. The study involved observation 
of users doing searches in an academic library, questioning them 
about what they had done, and then analyzing the system transaction 
log of those searches. The users' reports about their searches were 
then compared with the machine-recorded transcriptions of those 
searches. Major findings of the study include: (1) users reporting 
overload also reported finding a significantly smaller number of 
postings than other users and were liJcely to have experienced 
overload in the past, suggesting that they have information 
processing capacities or search styles that inhibit their 
persistence; (2) the median number of postings considered to be "too 
many" by all of the users was 15; (3) more users whose searches 
retrieved between 15 and 30 postings displayed records of all 
postings than ceased searching without displaying any records; and 
(4) users' persistence falls off significantly when the number of 
postings exceeds more than 30. These findings suggest thac surveying 
users about their preferences for the length of a list of references 
they are willing to scan on an online public information system will 
underestimate their persistence in actually scanning lists of 
references. They also suggest that system designers may defer 
providing help for users in coping with large number of postings 
until that number reaches 30. A copy of the user questionnaire is 
appended. (6 references) (BBM) 
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LCS Postings 
Executive Summary 



User persistence in searching for information is an 
important aspect of information-seeking behavior. Because much 
information-seeking is discretionary the user faces no penalty 
for failure to find information sought too long a set of 
postings may cause the user to abandon a search. Discussions of 
design of online information systems have recognized ^this 
possibility and have recommended that systems help users once the 
number of postings their searches retrieved exceeds a certain 
level. But none of these recommended levels is based on 
research . 

This study seeks to determine what levels of persistence in 
scanning postings are typical of users of one online public 
information system, LCS (Library Computer System), as it was 
configured in Illinois in 1987. The study observed users doing 
searches, questioned them about what they had done, and then 
analyzed the system transaction lug of those searches. By 
combining analysis of what the users reported about their 
searches with analysis of the machine-recorded transcription of 
those searches, the investigators compared userfs' reports with 
their behavior. 

Major findings of the study are: 
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1) Users reporting overload also reported finding a 
significantly smaller number of postings than other users; 
furthermore, users reporting overload were likely to have 
experienced overload in the past. Both these suggest that 
overloaded users have Information processing capacities or search 
styles that inhibit their persistence* 

2) Among all users responding to the questionnaire (both 
those reporting overload and those not reporting overload) the 
median number of postings considered to be "too many" was 15, 

3) Although 15 postings was generally considered to be too 
:nany, among users whose searches retrieved between 15 and 30 
postings, more users displayed records of all postings than 
ceased searching without displaying any records, 

4) Users' persistence falls off significantly when the 
number of postings exceeds 30. When a search retrieves 30 or 
fewer postings, a majority of users displays all postings. When 
a search retrieves more than 30 postings, a majority of users 
displays no records. 

If these findings can be generalized to other systems, they 
suggest that surveying users about their preferences for the 
length of a list of references they are willing to scan on an 
online public information system will underestimate their 
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persistence in actually scanning lists of references. The 
findings also suggest that designers of systems may defer 
providing help for users in coping with large numbers of postings 
until that number reaches 30. 

The findings are limited because LCS contains a design 
feature that seems to inhibit users from successfully persisting 
in their searches. Instead of dis'p laying postings by a series of 
numerically consistent commands, the user displays postings by 
issuing commands of PG2, PG3, PG+, PG2, PG3, PG+, etc. Review of 
the transaction logs shows that many users do not employ the PG+ 
command correctly, but use PG4, PG5, and the like. (The present 
study's analysis of the transaction log compensated for this 
design feature . ) 

Despite this design feature, the version of LCS used is good 
for a study of persistence because it displays postings in no 
apparent order. In this sense, it provides a baseline against 
which to compare users responses to other systems that display 
postings in various types of helpful order, (e.g., alphabetical 
and reverse-chronological). 



ERLC 



9 



Persistence in Scanning 
1*0 Introduction 



LCS Postings 



9 



How long users are willing to persist in scanning displays 
of postings is an important question for designers of machine- 
dri ven information systems. The term 'costings means in some 
contexts the number of records that a search retrieves; in other 
contexts, it means the records themselves. A system may display 
the number alone or it may display the number in combination with 
one or more full or truncated postings or with a guide to the 
postings. If all the full or truncated matches cannot be 
displayed at once, the user faces a decision about expenditure of 
nime: is it worth the time to look at additional postings to 
find the information sought? Because, as Wiberley and Daugherty 
have argued, much information seeking is discretionary users 
may abandon a search without fear of penalty too large a 
number of postings may influence a user to stop searching. 1 
Discussions of information-system design sometimes suggest a 
number of postings beyond which the system should provide 
prompts, helpful orders, and other mechanisms to help users cope 
with the postings. 2 None of these numbers, as I'ar as we can 
tell, derives from research. 

Presumably each time a user conducts a search, he or she is 
willing to persist in scanning a limited number of postings. 
(This limited number is not necessarily a small number.) It 
seems fair to assume that a given user will have a different 
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Persistence in Scanning LCS Postings 10 

limits in different situations. And across a population of 
users, limits will vary from person to person as well as from 
time to time. But for the designer of an information system, it 
would be useful to kno\/ the range of persistence. This is 
particularly true of information systems like online public 
access catalogs (OPACs) which are open to all persons and do not 
require sign-ons or passwords that right load into the system a 
profile of the user's tendencies and proficiency. Such systems 
must meet the needs of the public in all its variety. 
Heretofore, there has been no research reporting users' behavior 
on which to base design of system features that help users deal 
with large numbers of postings. This report describes a study 
that attempted to determine the thresholds of persistence for 
users of one information system, LCS (Library Computer System), 
as it was configured in Illinois in 1987. 

At the time this study was conducted, LCS (since enhanced) 
was an online library system providing author, title, 
author/title, and call number access to library collections. 
This investigation of use of LCS was conducted in an academic 
library that had loaded approximately 425,000 records into the 
system at the time of the study. While users had access to more 
than 25 other databases the largest of which was approximately 
3,300,000 records, 80 percent of the searches were conducted on 
the local database. The overall plan of the study called for 
observing when a representative sample of users began and ended 

ERJC .i 1 
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their sessions at LCS terminals. After the sessions, project 
staff gave these users a questionnaire about their experience in 
searching the system, asking particularly if the number of 
postings found was too many. The questionnaire asked about past 
use of LCS and other computer systems, and requested personal 
profiles, like those gathered for the Council on Library 
Resources (CLF) sponsored Online Public Access Catalog (OPAC) 
project. 3 The; investigators developed the questionnaire with the 
assistance of the University of Illinois Survey Research 
Laboratory (SRL). It was pre-tested on SRL employees and library 
users. Finally, the investigators analyzed transaction logs of 
the sessions to verify the users' recollections and to note other 
characteristics of user behavior. 

This study has cast the problem of overload in terms of 
numbers of postings instead of either number of screens of 
display or elapsed time of searching. While the latter two may 
ultimately prove more meaningful frames of analysis, numbers of 
postings is best for an initial investigation of the problem 
because system designers have discussed the problem of overload 
in terms of number of postings. 

2.0 Sampling Plan 



The sampling plan followed the model designed for the OPAC 
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project sponsored by CLR.^ User sessions were observed and 
questionnaires distributed at terminal locations — sites of a 
single terriinal or cluster of minals. Terminal locations were 
studied on a scheduled basis, each location at least once. The 
data collection period was the ten-week academic quarter of 
winter 1987 (5 January - 13 March 1987). Each day of the quarter 
was divided into three four-hour time blocks: morning, 9 a,m,-l 
p.m.; afternoon, 1 p.m,-5 p.m.; and evening, 5 p.m. -9 p.m. Data 
on numbers of transactions for January through March i986 showed 
that the morning and afternoon time blocks for Monday through 
Friday were the ten highest periods of activity. Observation 
during these ten time blocks was then assigned to terminal 
locations in proportion to level of activity; the more heavily 
used a location, the more time ^^lloted for observation of 
sessions and questionnaire di - ^bution at that location. Thus, 
a site of three terminals near the reference desk had the most 
activity and was subject to more hours of observation than any 
other site. Assignment of time blocks for observation at the 
different sites was done by random sampling -echniques. 



Research assistants obse rved the beginning and ending of the 
first user-session in a time block, and, at the session's end, 
gave the user a questionnaire. The assistants then repeated this 
process for each succeeding user whose entire session could be 
observed. Thus, assistants did not question users who began 
their sessions when another user was being observed. In short, 
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users sampled were individuals who were observed by the research 
assistants, worthing at an LCS terminal at a designated terminal 
location during tht data collection period during the randomly 
assigned time block5>. E eluded from this study were individuals 
using the system via dial access from remote terminals. 

The target sample size was 1,300. This assumed five usable 
sessions per hour, an assumption based on observation of activity 
at terminal locations in spring 1986 and a response rate 
comparable to that in the CLR studies. The total number of time 
blocks assigned was 78 or 312 hours. Of the users sampled in the 
CLR studies, about 17% reported finding too much i n f o rma t i on . 5 
Based on these data, the present study expected to cover 221 
cases where too many matches were found. A factor which was 
expected to have an effect on response, but whose extent could 
not be projected was migration of users from LCS to LUIS, the 
Lib rary s NOTIS—based OPAC. When the study team observed 
activity at terminals in spring 1986, LUIS had been available to 
the public for only a month. In the following seven months the 
library 's clientele had gained increasing familiarity with LUIS, 
and many users were attracted to the newly-installed system. 



3.0 LCS Displays 



LCS gives users access to a file of bibliographic records 
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through general searches and detailed searches. The user 
constructs general search keys from elements of the author^s name 
and/or the title of the item sought. The system responds with a 
listing of any items matching the search key. (See Figure 1.) If 
no items are retrieved, there are no listings. If only one item 
is retrieved, the system displays a detailed record (described 
below). If more than one item is retrieved, the system displays 
general records. Each general record is one line long and shows 
author, title, and publication place and date. Author, title, 
and place may be truncated to fit the single line. Up to ten 
general records are displayed at one time in units called pages. 
The user may page through the postings, ten at a time, or may 
skip ahead by specifying the number of matches to skip. Because 
matches are not displayed in any apparent order, however, the 
user has no basis upon which to guess how far into the listing a 
desired item may fall. At the top of the first page, the system 
displays the number of items matching the search key so the user 
knows at the outsec how many postings are retrieved. 

Detailed searches are typically constructed with a line 
number of a posting displayed in a general search or by using the 
call number, if it is already kaown. The result of a detailed 
search is a detailed record that includes the czll number, author 
and title, place and date of publication, holdings, location, 
date added to database, and circulation information (i.e., 
availability.) (See Figure 2.) 
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Figure 1 

General Search Statement and General Records 



AUT/GRABERDOR 



General Searah Statement 



PAGE 1 

01 GRABER, 

02 GRABER, 

03 GRABER, 

04 GRABER, 

05 GRABER, 

06 GRABER, 



6 MATCHES 



DORIS 
DORIS 
DORIS 
DORIS 
DORIS 
DORIS 



(DORIS 
(DORIS 
(DORIS 
(DORIS 
(DORIS 
(DORIS 



0 SKIPPED 



(ALL DISPLAYED IN' 1) 



APPEL), VERBAL BEHAVIOR AND POLITICS $URBANA 1976 

APPEL), MASS MEDIA AND AMERICAN POLITICS $WASH 1980 

APPEL), PROCESSING THE NEWS $NEW YORK 1984 

APPEL), CRIME MEWS AND THE PUBLIC $NEW YORK 1980 

APPEL), PUBLIC OPINION, THE PRESIDENT, AND FOR 1968 

APPEL), MASS MEDIA AND AMERICAN POLITICS $2D 1988 

General Records ••' 



Figure 2 

Detailed Search Statement and Detailed Record 



DSL/3 



Detailed Search Statement 



ffi1261G781984 GRABER, DORIS A. (DORIS APPEL), 1923- 
PJJOCESSING THE NEWS $NEW YORK 83-10537 

388409 1984 1 ADDED: 840614 

01 001 3W MAI RNEWD 860501/870222 CC 

detailed Record 
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Because users do not indicate when they begin or end LCS 
sessions, project staff observed the sessions. Staff kept an 
observer log and recorded (1) the beginning time of the session 
(2) the ending time of the session; (3) the terminal ID of the 
specific terminal used; and, (A) the gende^ of the user. 
Observers used clocks synchronized with LCS at the start of the 
project; synchrony was verified during the survey. There were 
several challenges in noting exactly when sessions began and 
ended. In order to be unobtrusive, observers were stationed 
between eight and thirty feet from the terminal sites. Traffic 
between them and the users could distract them from observing 
exactly when users first issued a search command. Noting the end 
of a session was complicated by two factors. First, the observer 
did not know a session was Dver until the user left the terminal, 
but the user might remain at the terminal for some time after 
issuing a final command. Second, the observer had to distribute 
the questionnaire as soon as the user left the terminal. 

Comparison of the system transaction log and the observer 
log revealed discrepancies between the two for many sessions. 
Because of these discrepancies, observed sessions in the 
transaction log were identified by scanning the log for the 
terminals and times observed. To facilitate correlation of the 
transaction log with the observer log, the chronological 
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transaction log was sorted, first, by terminal ID and then 

chronologically for each terminal. Terminal IDs and beginning 

and ending times from the observer log were programmed to 
interfile in the transaction log. 

Only 81 of 748 (11%) observed sessions could not be 
correlated with transaction log activity. Generally correlations 
were straightforward. In these cases of correlation, the 
transaction log recorded a discrete block of activity that began 
exactly at or within two minutes of a session beginning time 
recorded by an observer. This block of activity was either 
separated by three or more minutes from other activity or it was 
distinguished by its content, that is, search keys were repeated 
or search keys apparently seeking the same authors or titles were 
used . 

Careful inspection of the transaction log and comparison of 
it with the observer log indicated the occasional dif f icu 1 1 i<5s 
the observers had in noting when users first and last entered a 
command in their sessions. There were 14 cases where the 
observer apparently recorded a beginning time for a session as 
soon as the user arrived at the terminal, but the user did not 
enter a command for three to six minutes. Also, there were 28 
cases where the absence of activity in the transaction log 
suggested that a user lingered for three to six minutes at a 
terminal without entering a search key, leading the observer to 
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record an ending time several minutes after the user had issued 
the last command of the session. 



There were 41 cases where the observer entered the wrong 
terminal ID into the log. These errors were identified when no 
discrete activity could be found at the terninal recorded, but 
transactions fitting the criteria for straightforward 
correlations outlined above, were identified at an adjacent 
terminal. There errors were understandable because terminal IDs 
were easily confused, e.g. number sequence in one location was 
left to right, in another right to left. Future projects might 
better use a graphic representation on observer log forms and ask 
observers to circle an icon of the terminal in use, rather than 
circle the ID. 



Also, in 15 cases continuity of the transaction log (at 
least one command every minute and all commands repeating or very 
closely resembling each other) indicated the observers had 
recorded starting times in the middle of sessions clearly already 
in progress. While the observers were not supposed to record 
sessions already in progress, the investigators saw no reason to 
eliminate the beginnings of sucH sessions from the data set when 
the content of the transaction log demonstrated the continuity of 
the sesbion. 



Finally, there were cases where the transaction log showed 



^9 



Persistence in Scanning LCS Postings 19 

search commands occurring more than two minutes after the 
observed end of the session, that repeated or very closely 
resembled search commands that were executed during the observed 
session. If such continuity of content was also accompanied by 
temporal continuity (at least one command per minute and the 
sequence of commands beginning within two minutes of the observed 
end of the session), the end of the session was identified as the 
end of this continuity. 

5.0 Editing the Transaction Log 

Program documentation for LCS available at the outset of the 
project stated that the transaction log recorded both the user's 
input and the system's respon^o of the number of postings 
retrieved. That is, if the usei conducted an author search for a 
book by Doris Graber (see Figure 1) the transaction log would 
record the search and that six matches or postings were 
retrieved- In fact, the transaction log did not record the 
number of postirtgs retrieved. The absence of these data forced 
the project team to reconstruct all searches that might have 
retrieved matches. In addition, since this reconstruction took 
place more than a year after the observed searches, it was 
necessary to perform detailed searches on each record retrieved 
to determine when it was added to the database. The number of 
records added after the date of the observed session was 

o CO 
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subtracted from the total found in the reconstruction. The 
corrected totals, then, were the same as those seen by the users. 
These totals were used in the analysis of the transaction log. 

Some records were also deleted from the da abase between the 
months when sessions were observed and the time when the searches 
were reconstructed. Although it was not possible to correct 
reconstructed searches to compensate for deletions, the number of 
deletions was estimated to be so small that it has little effect 
on this study's results. Library staff at the site library who 
edited their database reported that they normal ly deleted fewer 
than 40 records each month — less than one hundredth of one 
percent of the database. We have no reason to believe that other 
libraries' databases had significant deletions. 

6.0 Methods of Data Analysis 

This study worked with two kinds of data: (1) questionnaire 
data that included the user's (a) report about what happened in 
the observed session and (b) response to questions about his or 
her academic background and past use of computers, libraries, and 
library systems; and (2) transaction log data that provided a 
system-recorded transcription of the user's activity on the 
system that was supplemented by the investigators with the number 
of postings retrieved by correctly formatted searches. Each kind 
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of data was analyzed separately, and then the two data sets were 
combined and analyzed. 

6.1 Method of Analysis of Questionnaire Data 

Determination of (1) limits of persistence and (2) preferred 
number of postings, from questionnaire data alone was 
straightforward. All responses were keyed to disk using the 
software package, PC-Enter . Then they were uploaded to a 
mainframe and analyzed using SPSS-X, version 3.0 installed 
December, 1987 and operating on an IBM 3081K, running VM/CMS. 
Frequencies, cross-tabulations, and correlations were run as 
appropriate for the types of variables. 



6.2 Method of Analysis of Transaction Log Data 

Determination of levels of persistence and of overload from 
transaction log data alone is an inferential process, far more 
complicated than analysis of questionnaire data. Transaction log 
analysis depends on a set of assumptions about what constitutes 
normal user behavior. We assume, as do other catalog studies, 
that users search a catalog to find a call number of a known ite 
or a call number to serve as a starting point for shelf searchin 
for books about a subject. ^ Because call numbers are found in 
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detailed LCS records, this assumption means that, with LCS, a 
user who is not overloaded will find and display at least one 
detailed record if his or her search retrieves the known item 
sought or an item that the user thinks is about the subject 
sought. We assume that if a search does not retrieve an item a 
user seeks, then the user who is not overloaded will exhaust all 
possibilities by displaying general records for all items 
retrieved • 



Given the design of LCS, an investigator can infer 
persistence or overload only in searches that retrieve more than 
ten postings. No inference can be drawn for searches that 
retrieve ten or fewer postings, because the system automatically 
displays up to ten general records when retrieved. For searches 
that retrieve more than ten postings, the user can display 
general records in groups of up to ten, until general .^ecords are 
displayed for all postings retrieved. For example, if a search 
retrieves 37 postings, the system automatically will display 
general records for postings 1-10; the user can then choose to 
display general records for postings 11-20, 21-30, and 31-37. 
For a user who displays postings 11-30 and displays no detailed 
record, we can infer a level of persistence of 30. Similarly, 
for a user who displays all 37 postings, we can infer a level of 
persistence of 37. 



For a user whose search retrieves more than 10 postings, but 
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issues no command to display more general records and nc command 
to display a detailed record, we can say only that his or her 
reaction to this number of postings exhibits overload. 
Similarly, users who display some, but not all, general records 
retrieved also exhibit overload for the level of postings 
retrieved. But members of the latter group also exhibit levels 
of persistence equal to the number of postings each displays. 
This would be the case for the person who displays 30 of 37 
records retrieved. 

The designer of a public information system must ask at what 
level a system should help users cope with the number of postings 
retrieved because it is too large. We would posit this level or 
threshold for help should be the number of postings above which 
more users exhibit overload than exhibit persistence. In 
spelling out what this means, we point again out again that no 
inference can be drawn for searches that retrieve ten or fewer 
postings, because this system automatically displays up to ten 
general records when retrieved. Thus, at each level of postings 
above ten, the number of overloaded users (those who do not 
display any detailed records nor any general record) must be 
compared with the number of persistent users (those who display 
all general records or some general records but no detailed 
records ) . 

In interpreting the transaction log, the investigators took 
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into account the structure of commands for displaying general 
records. Instead of displaying general records by a series of 
numerically consecutive commands, the user displays these records 
with commands of PG2, PG3, PG+, PG2, PG3, PG+, etc. for records 
11-20, 21-30, 31-40, 41-50, 51-60, 61-70 respectively (1-10 are 
automatically displayed). Figure 3 illustrates use of the PG+ 
command to display postings 31-37 in a search that retrieves 37 
postings. Because the transaction log strongly suggested that 
many users attempted to persist but were unsuccessful in efforts 
to display more general records because they issued incorrect PG 
commands, the investigators decided to develop rules to give 
credit for such effort. First, users received credit when they 
issued a PG command with an illegal number (since given the 
results of using PG2 and PG3, it is reasonable to assume PG4 
displays general records 31-40; PG5, 41-50, etc. Second^ c red i t 
was given for the first illegal repetition of a PG+ command, 
because it would be reasonable to assume PG+ displays in groups 
of ten all records beyond 30, until illegal use of the command 
shows this is not always the case. We/^all an interpretation of 
the transaction log that gives users credit for such incorrect 
commands that are apparent efforts to display additional general 
records a liberal interpretation. In Tables 1-4 a liberal 
interpretation is designated by an open circle (o) in contrast to 
a solid circle (•) where the user issued a correct series of 
commands to display general records. 
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Figure 3 

Commands that Display General Records 



TLS/CLINCHEMI 



37 MATCHES 



PAGE 1 
01 
02 

03 GRAY, C. H. (CHARLES HORACE) 

04 SYMPOSKJM ON THE CLINICAL CHEMIS 
05 

06 ZILVA, JOAN FOSTER 

07 HENRY, RICHARD JOSEPH 

08 BOUTWELL, JOSEPH H. 

09 GRAY, CHARLES HORACE. 

10 KANTER, MURIEL W. 



PG2 



37 MATCHES 



PAGE 2 

11 ZILVA, JOAN F. 

12 HENRY, RICHARD J., 1918- ED. 

13 GRAY, CHARLES HORACE. 

14 REYNOLDS, MOIRA DAVISON. 

15 STEWART, CORBET PAGE. 
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After Llie two i n vos t i ga I o r s most fnmiliar witli LCS had 
(^s I n l> I i sIkmI I Ih' iiuH hod (l(\*^(Mh(Ml h(M(^ foi i n ( (m r i n g l(^V(Hs of 
persistence and ovoi lofid I rom I he transaction log, they analyzed, 
i ndepeiulenM y , I lie sognonls of the Iransaction log for each 
session observed. They then compared these independent analyses 
and reconciled by catefnl r(^vie\sr of I be log, any dirferencos in 
the independent analyses. 

T n some of the o b s e r v e I sessions, u s e r s' s e a r c h e s retrieved 
more than ten postings more than once. For this report, the 
investigators analyzed the user's reaction to only the highest 
number of [)Ostings retrieved. This approach assumes that 
persistence in displaying the higher number is more noteworthy 
and that users presumably had the hi. ghest number of postings in 
mind when answering (juestions about whether they found too many 
in the session. 

Tn some sessions, searches retrieved more than once the 
highest number of postings found in the session. Fn these cases 
the investigators analyzed (and re[)ort in Tables 1-A below) the 
user's response that showed the most persi stence . For example, 
in one session immediately after a search key retrieved 32 
postings, the user tried different search keys that retrieved 
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smaller numbers of postings, then reentered the search key that 
retrieved 32 postings, and finally displayed general records of 
all 32 postings. The investigators classified this as a case of 
total persistence in scanning 32 postings. While the user may 
not have persisted initially, he or she did try alternate search 
keys (a variant manifestation of persistence), and, ultimately, 
within the same session, did persist in scanning the 32 postings. 



6,3 Method of Analysis of Questionnaire and Transaction Log Data 
Together 

For cases where users responded to the questionnaire, the 
investigators examined the transaction log to compare users* 
behavior with their reports about what happened during their 
sessions. This comparison focused on how the user reacted to the 
largest number of postings retrieved during the session. As 
stated above, users presumably had the largest number of postings 
in mind when they answered questions about whether they found too 
many postings. After interpreting the transaction log to 
identify cases where the users* reactions were either of overload 
or of total persistence to the largest number of postings 
retrieved in a session, the investigators then compared the 
characteristics, ascertained by the questionnaire, of users in 
the two groups. 
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7.1. Results of Analysis of Questionnaire Data 

The analysis of questionnaire data showed that 418 of the 
748 users who were observed returned a completed questionnaire (a 
return rate of 56%). Of these 418, 44 or 11% reported they 
experienced overload during the observed sessions. Analysis of 
these users' responses shows that 13 postings was the median 
number reported to be too many. Of all respondents, 35% reported 
that they had experienced overload at some time. Two hundred 
seventy-two users or 65% (both those who had and some who had not 
experienced overload) responded to the question, "In general, how 
many matches [postings] would you consider to be 'too many.'" 
The median response was 15 postings. 

Search overload is significantly correlated with a lower 
number of postings found. This suggests that overloaded users 
may have lower thresholds. Is this overload specific to a 
particular search, or is it a more general user characteristic? 
We found that specific overload is significantly correlated with 
having ever experienced search overload (r = ,30, p < .001), The 
specific and prior number of postings are also positively 
correlated (r = .63, p < .001). This suggests that overload 
potential is to some extent a general user trait. It could be 
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evidence. The analysis includes cases of these who did not 
respond to the questionnaire because their activi*:/ can be 
analyzed in the same way as the activity of respondents. 



In the tables, reactions of users are divided into three 
categories. The category of users in column (2) is those who 
presumably were overloaded by the number of postings they 
encountered: they displaved no general records after the system 
finished automatically displaying the first ten and they 
displayed no detailed records. Column (3) gives data on users 
who showed total persistence by examining all possibilities among 
the postings they retrieved by displaying all general records 
beyond the first ten. They were totally persistent. Column (5) 
also shows extent of persistence of users who displayed some, but 
not all, of the general records retrieved by their searches. 
The persistence of users reported m column (5) is comparable to 
that in column ( 3) • 



The patterns of reactions to different levels of postings in 
relation to other data found in this study suggest the following 
recommendations for designers of information systems. In 
relation to users' reports of their preferences for limits in 
numbers of postings, the median expressed preference of 15 
appears to be significantly lower than the number with which manv 
users will work. Of 83 users in Table 4 whose searches retrieved 
15 or more posuings, 47 (57%) in some way persisted by displaying 
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more than 15 general records. Of the 47 cases in Table 4, where 
searches retrieved between 15 and 30 postings, only 14 (30%) 
displayed no general records. A majority of users do not exhibit 
overload until the number of postings retrieved exceeds 30. The 
need to use PG+ to display postings beyond 30, may inhibit 
persistence beyond that level. Liberal interpretation of the 
transaction log tries to compensate for this, but study of users* 
persistence on other systems where it is easier to display more 
than 30 postings will be needed to test this finding. 



7.3 Results ot Analysis of Questionnaire Data and Transaction 
Log Data Combined 

Analysis of the transaction log and questionnaire data 
together offers a valuable perspective on user persistence. 
First, 11 of the 44 (25%) users who reported finding too many 
postings (i.e., were overloaded) did searches that retrieved 
either nc postings or only one posting, and thus saw everything 
their searches had to offer. Their reports do not fit a common 
sense understanding of the meaning of too many postings. This 
suggests possible confusion in use of the system or 
misunderstanding of the questionnaire. Second, of the 18 users 
who reported overload and whose searches retrieved more than ten 
postings, five (28%) displayed all general records. This 
suggests that some users feel overloaded, but it does not prevent 
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them from persisting through a list of postings they consider too 
long. 

Some users who reported they were not overloaded also 
behaved in a way not consistent with their report about what they 
had done. Of the 53 who did not report overload, whose searches 
retrieved more than ten postings, and who were not partially or 
totally persistent, 19 (36%) did not display any detailed records 
nor any general records after the first ten that the system 
displayed au toma t i':al 1 y . Another four displayed some, but noc 
all of the general records after the first ten and no detailed 
records. By not seeking a detailed record and by not displaying 
any additional general records, these users were abandoning their 
searches before exhausting all possibilities* 

Analysis of questionnaire responses of users who either 
reacted as if they were overloaded (i,e,, displayed no records) 
or totally persisted (i.e., displayed all general records) 
revealed no differences between the two groups, 

8 • 0 • Con c lusions 

The study's findings suggest that surveying users about 
their limits in scanning a list of references on an online public 
information system will identify a median limit that 
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Table 1 

Reactions of Overload and Levels of Persistence 
Among Users Who Reported Overload 
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Table 2 

Reactions of Overload and Levels of Persistence 
Among Users Who Did Not Report Overload 
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Table 3 

Reactions of Overload and Levels of Persistence 
Among Users Who Did Not Respond to the Questionnaire 
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Table 4 

Reactions of Overload and Levels of Persistence 
Among Users Who Reported Overload, Did Not Report Overload, and Did Not RespDnd to the Questionnaire 
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underestimates the persistence of a majority of users. Although 
15 is the median number of postings' that users say is too many, 
when searches retrieved betwetn 15 and 30 postings, more users 
displayed records of all postings retrieved than ceased searching 
without displaying any records. 

Because system intervention to help users cope with overload 
can degrade system performance in other areas, designers of 
information systems will want to defer helping users with large 
numbers of postings as long as possible. The findings of this 
study suggest most users do not need such help until the number 
of postings exceeds 30. Users' persistence falls off 
significantly when the number of postings retrieved exceeds 30. 
V/hile a majority of users displays all general records for 
searches that retrieve between 11 and 30 postings, when searches 
retrieve more than 30 postings, a majority of users displays no 
records . 

One might argue that, although a majority of users displays 
records for all postings when a search retrieves between 15 and 
30 postings, a substantial proportion (14 of 47, or 30%) do not 
display any records and the needs of this minority should be 
recognized. There are two arguments against this. First, as 
stated above, provision of help is costly and should be minimized 
to save resources when possible. Second, analysis of the self- 
reported characteristics of users suggests that those who are 
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overloaded are overloaded at very lov levels of postings, and, 
even when they apparently know how to reduce the number of 
postings retrieved, are unwilling to do so. 

The findings of this study are limited because LCS's design 
seems to inhibit some users from successfully persisting in their 
searches. While the study's analysis of the transaction log 
attempted to compensate for LCS design, investigation of user?^* 
reactions to other systems will be needed to confirm this study's 
findings. At the same time, the findings of this study serve in 
one way as a baseline for investigations of persistence: unlike 
other systems that display postings in helpful orders (e,g, 
alphabetical by author or reverse chronological), this version of 
LCS displays postings in no discernible order. 

The need to understand users' persistence in scanning 
postings in machine-readable information systems will always be 
with us, and probably will increase in significance. The trend 
is to build larger and larger databases and to make each word m 
r:he database retrievable. This will result in databases yielding 
more and more search results of large numbers of postings. The 
need to provide users with assistance in coping with these 
instances of overload will similarly increase. The findings of 
this study will be modified and extended by future research. But 
until that the research is done, the results reported here can 
serve as a guide for system design. 
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USER QUESTIONNAIRE 
This is a survey on the use of the Library Computer System (LCS) and 
how effectively it meets your needs. Please take a few minutes to 
answer the questions; circle the number next to your answer. Return 
the completed questionnaire to the box provided. 

In any of the searches you just performed, did you find too many matches? 

Yes 12.2% 
No 316 87.5 
? 1 0.3 
NA 57 

(if NO, skip to 
question h) 



In the searches where you found too many matches, what number of matches 
were there? 
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In that search, did you search using the author's name, 22 19.8% 

the title, 30 27. 0 

both author and title, 32 28.8 
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subject, or U 12.6 

other (please specify) 13 11. 7 
Don't know 

No Answer (NA) 307 
using this system, do you ever find too many matches? 
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general, how many matches would you consider to be "too many?" 
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1000 I 
2000 1 
NA 146 
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99.6 
100.0 



6. 



In using this system, how often do you find what you're looking for? 

Less than half of the time 115 32.7% 

More than half of the time 237 67-3 
No Answer (NA) 66 

Here are some quesLior.c abouty how often you use libraries and library systems, 



7. 



Of the following, which comes closest to describing how often you use a 
library — any library at all? 
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8. 



Of the following, which comes closest to describing how often you use this 
system (LCS) by dial-access, ADN, or in a library? 
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9. 



In general, which comes closest to describing how often you use computer 
systems other than these library systems? 
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il» 


3.9 


Don't know 


U 


3.9 


No Answer (NA) 


59 





Here are some questions about the display of multiple matches in this system. 

10. Where more than one match is displayed, would you say that scanning 
through the display is easy, or 273 78.4* 

difficult? 75 21.6 
NA 70 

11. Would you say that the matches were displayed in an order that is 

clear 279 79.9* 

not clear? 70 20.1 



ERIC 



r 
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NA 



69 



12, Would you say that the displays are in a format that is 

easy to understand, or 260 

difficult to understand? 91 

No Answer (NA) 67 



7k. ]% 
253 



13- Do you know how to reduce the results when there are too many matches? 

Yes 166 k7.0% 



In this session, what 



No 


186 


52.7 


NA 


65 




was the main purpose of you library 


research? 


Was it for a course you are takirig. 


155 


k2.7% 


a course you are teaching, 


1 1 


3.0 


thesis or dissertation research. 


56 


15. 


independent research, or 


7^ 


20. k 


recreat iona 1 readi ng? 


18 


i».6 


Other: 


^^9 


13.5 



15« Of the categories shown here, which best describes your current status? 



Freshman/Sophomore 69 

Junior/Senior 124 

Graduate (Masters) 75 

Graduate (Doctoral) hk 

Graduate (Professional) 6 

Faculty 30 

Staff k 

Other: 13 

Don' t know 1 



19-0% 
33-9 
20,5 
12,0 
K6 
8,2 
M 
3-6 
0-3 



16. Of the categories shown here, which best describes you academic area? 



17 • In wi'.at year were you born? 



Arts & Human! ties 




77 


21.0% 


Business 




50 


13.7 


Education 




9 


2.5 


Engineeri ng 




81 


22. 1 


Law 




6 


1.6 


Medical /Health Sciences 


21 


5.7 


Physical/Biological 


Sciences 


29 


7.9 


Social Sciences 




51 


13.9 


1 nterdi sc ipl i nary 






0.3 


Major not declared 






1.9 


Other: 




31 


8.5 


Don't l<now 






0.3 




1923 




0.3* 




192'» 




0.3 




1926 




0.3 




1935 




0.3 




1937 




0.9 




1938 




0.3 




1939 




0.3 




19'>0 


2 


0.6 




19'»1 


2 


0.6 




9 



r 



• 
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19^2 


■J 




19^43 


2 


0,6 






0. Q 


19^45 


2 


0,6 


19^46 


1 


0. ^ 








19^48 


1 


0. ^ 




J 


0 Q 




6 


1,8 




7 


7 1 


19'?2 


7 


7 1 


1 


L 
•t 


1 7 


195'4 


10 




19<)<) 


Q 


7 6 


1 


1 2 






1 "t 








^ 8 




q 


2 6 


1 960 




0 • ^ 


1961 




7 


1 962 




8 ^ 


196^ 




•t • "t 


196^ 




8 <; 




2^ 


7 1 


1966 


22 




1967 


18 


5,3 


1968 


3^ 


10,0 


1969 


5 


^5 


^971 


1 


0-3 


198'» 


1 


0-3 



This is the end of the questionnaire. Thank you for your participation. 
Please place the questionnaire in the box provided. 



50 



